Image data coding and decoding system for efficiently compressing information using the shape and position of the image content

ABSTRACT

An image data coding system comprises a screen-area determining means for determining the area of a screen reproduced on the basis of an input image data; a code-amount assigning control means for controlling regions to which data on the screen are to be assigned and the code amount assigned to each of the regions, on the basis of the results determined on the area of the reproduced screen; and a coding means for coding the image data signal inputted in accordance with the code amount assigned to each of the regions. In an image data coding and/or decoding system, a coding system performs the two-dimensional orthogonal transform of picture signals of all the pixels with respect to inside blocks and of only picture signals of pixels contained in a content with respect to edge blocks, in accordance with a map signal indicative of the position and shape of the content, and it codes the map signal. A decoding system selects an orthogonal transform coefficient necessary to reproduce an image of a desired resolution from coded orthogonal coefficients on the basis of a coded and resolution-transformed map signal. It performs the two-dimensional inverse orthogonal transform of all the coefficients with respect to the inside blocks and of only the coefficients contained in the content with respect to the edge blocks thereby deriving a resolution-transformed regenerative signal.

This is a continuation of application Ser. No. 08/554,916, filed Nov. 9,1995, now abandoned.

BACKGROUND OF THE INVENTION

The present invention relates generally to an image data coding and/ordecoding system which can carry out high-efficient coding of picturesignals to transmit and store. More specifically, the invention relatesto an image data coding system which can code and transmit picturesignals to display an image on a liquid crystal display with a smallscreen which can be built in a wristwatch and so forth.

In the coding of image data used for a visual telephone (TV phone), atelevision conference and so forth, the image data efficientlycompressed utilizing human's visual characteristic are used. The human'svisual characteristic with respect to the distortion of a pictureutilized here are as follows (see "Image Information Compression",issued by Japanese Television Society and complied under the supervisionof Hiroshi Harashima, page 12).

(1) Frequency Characteristic in Distortion Perception

Distortion varying with elapsed time and distortion with high spatialfrequency are difficult to be visible to the naked eye.

(2) Relationship with Pattern of Image

Distortion is easy to be perceived at the flat portion of the image, anddifficult to be visible on the contour portion of the image. However,this is the case of a still picture. In a moving picture, the distortionon the contour portion serves as an edge busyness to conversely offendthe eye.

(3) Relationship between Image and Motion

When a picture is moving at a higher speed than a given speed and theuser's eyes can not follow its motion, the perception sensitivity todistortion lowers.

(4) Relationship with Switching of Scene

Immediately after a scene has been switched, the distortion is not to bevisible to the naked eye if the resolution considerably lowers

(5) Relationship with Brightness of Screen

The more the screen is dark, the more the picture distortion of the samelevel is easy to be visible to the naked eye.

(6) Color Signal and Luminance Signal

Since distortion by color signals is more difficult to be visible to thenaked eye than that by luminance signals, for example, it is possible tothin out sampled points of the color signals.

In addition, since visual acuity (spatial resolving power) on theperipheral portions of the visual field is worse than that on thecentral portion thereof under the influence of the distribution ofvisual receptor cells on retinas, it is necessary for a user to move hiseyes (eye movement) in order to obtain information such as shape,structure and detail contents (see "Image Information Compression"issued by Television Society, published by Ohm, page 41). Therefore, todetermine the definition of the picture in view of human's visualcharacteristic is dominated by the movement of human's eye serving as asubjective factor in addition to the resolution of the picture servingas an objective factor.

On the other hand, when a human looks at an object, if the object issmall, it is possible to recognize the whole shape and so forth of theobject by staring a specific range around a point. However, if theobject is large, it is necessary to closely observe a wide rangeincluding a large number of points to recognize the whole shape and soforth of the object. When he watches a television receiver, if itsscreen is large, a large number of closely observed points aredistributed in a given range by frequently moving his eyes, but if thescreen is small, the range wherein the closely observed points aredistributed does not so extend.

It is disclosed in "Estimation Technique of Image Quality and ToneQuality" (edited Television Society and published by Shokodo, page 118)that since the display screen in a high quality television system whichrapidly approaches to implementation in recent years is greater thanthose of current television systems, the closely observed pointsdistributing ranges in these systems are different. FIG. 5.22 on thesame page of this paper shows the measured results of proportion of theclosely observed points distributing range to the area of the screenwhen observing a high quality television system and a current televisionsystem on a standard observation condition using a program of the samecontent. This figure is expressed by approximately to an ellipse withthree times as large as the standard deviation assuming that the closelyobserved points lie on a normal distribution in horizontal and verticaldirections when the center of the screen is the origin. It is also shownthe experimental results that the proportion of the distribution rangeof the closely observed points to the area of the screen is about 60% inthe current television systems, but it reaches about 80% in the highquality television system. That is, as the size of the screen decreases,the proportion of the distributing range of the closely observed pointsdecreases and the range concentrates on the center of the screen.Therefore, since the spatial resolving power of the visual sensation onthe peripheral portion of the screen is inferior, the informationcompression can be efficiently carried out by lowering the spatialresolution or by weighting the assignment of the distortion inpreprocessing.

By the way, as a method for efficiently compressing the measure ofinformation using the difference between the visual characteristic atthe central portion of the visual field (central vision) and the visualcharacteristic at the peripheral portion of the visual field (peripheralvision), there is a method disclosed in, for example, "Visual PatternImage Sequence Coding" (August, 1993, IEEE TRANSACTIONS ON CIRCUITS ANDSYSTEMS FOR VIDEO TECHNOLOGY, VOL. 3, NO. 4, pp-291-301). In thetechnique disclosed in this literature, a function relating to theposition of radius r from the central point of the screen is derived,and the resolution on the peripheral portion of the screen is loweredusing this function.

In addition, as a method for performing the information compression bychanging the distribution of the assigned code amount in a visuallyimportant region and an unimportant region, there are two methods asfollows.

One of the methods has been proposed as applied to a video telephone(Japanese Patent Application Laid-open No. 1-80185 (1989) "MovingPicture Coding Method"). In this method, on the assumption that theclosely observed points are concentrated on the face of the oppositeparty for the telephone conversation, the face region is detected toassign many code amount on the detected face region.

Another method is also applied to a video telephone similar to theaforementioned proposal (Japanese Patent Application Laid-open No.5-95541 (1993)). Similar to the aforementioned proposal, by detectingthe face region to apply a spatial-temporal filtering to a region otherthan the face, the code amount produced in this region other than theface is decreased, and the code amount assigned in the face region isincreased.

Both of these conventional methods pay attention to human's visualcharacteristic, and provide a natural picture to a person which visuallyrecognizes a reproduced picture, by changing the coded data amount sothat the observed points are concentrated in the distribution of closelyobserved points, is different from the coding data amount in the regionin which the closely observed points are not so concentrated.

As mentioned above, in both of the conventional image data codingmethods, the information compression has been efficiently performedusing human's visual characteristic by restraining the code amountproduced in a visually unimportant region and by increasing the codeamount assigned to a visually important region. However, both of thetechniques disclosed in the aforementioned two publications onlyclassify the regions in the screen on the basis of the degree ofconcentration of the distribution of closely observed points, to varythe code amount assigned to each of the regions, and these techniques donot consider human's visual characteristic that the distribution ofclosely observed points is different by the size (area) of the screen asdescribed in the aforementioned literature "Estimation Technique ofImage Quality and Tone Quality".

In addition, there are problems in that when the image data aretransmitted via a radio transmitting channel having a narrower bandwidththan that of a wire transmitting channel, the resolution of thereproduced picture is generally decreased by the limit of thetransmitted amount due to the narrow bandwidth, so that the size (area)of the screen is necessarily decreased.

By the way, in conventional image data coding systems, for example, inmoving picture data coding systems defined by MPEG, after inputtedpicture signals are divided into square blocks of 8×8 pixels as shown inFIG. 55, the two-dimensional discrete cosine transform (DCT) isperformed for coding.

On the other hand, in "Applying Mid-level Vision Techniques for VideoData Compression and Manipulation" (M.I. T. Media Lab. Tech. Report No.263, February 1994), which will be hereinafter referred to as"Literature 1", J. Y. Wang et.al. disclose that picture signals aredivided into a background and subject (which will be hereinafterreferred to as a "content") for coding, as shown in FIG. 56. Thus, inorder to code the background and the content separately, a map signalcalled a alpha map indicative of the shape of the content and itsposition in a screen is prepared. In this coding method, it is possibleto vary the picture quality content by content and to reproduce only aspecific content. However, as shown in FIG. 55, in a case where theinterior of a screen is divided into square blocks for coding, it isrequired to separately process the blocks containing the boundaryportion of the content, i.e. the edge blocks between the inside andoutside of the content, as shown in FIG. 57.

It has been also proposed a method for coding picture signals afterdividing the interior of a screen into blocks of optional shapes so asto adapt to statistical characteristic in the screen and to the shape ofa content. Such a method for performing the orthogonal transform of anoptional shape is disclosed in "Examination of Variable Block SizeTransform Coding of Image Using DCT" (Matsuda et.al.,Singaku-Shuki-Daizen D-146, 1992), which will be hereinafter referred toas "Literature 2". In this specification, this transform method will behereinafter referred to as "AS-DCT". In AS-DCT, first, one-dimensionalDCT is performed in a horizontal (or vertical) direction as shown inFIG. 58(a), and then, after it is rearranged in order of the low of theDCT coefficient as shown in FIG. 58(b), the one-dimensional DCT isperformed in a vertical (or horizontal) direction.

Also, in "Estimation of Performance of Variable Block Shape TransformCoding of Image Using DCT" (Matsuda et.al., PCSJ92, 7-10, 1992), whichwill be hereinafter referred to as "Literature 3", the coding efficiencyhas been improved by selecting the order of higher coding efficiency asa result of practical coding, as the order of the transform in thehorizontal and vertical directions.

Further, "Image Data Coding Techniques--DCT and ITS InternationalStandard--" written by K. R. Rao and P. Yip and translated by HiroshiYasuda and Hiroshi Fujiwara (7.3, pp. 164-165, Ohm), which will behereinafter referred to as "Literature 4", discloses a method forperforming the resolution transform of picture signals using thetwo-dimensional DCT. That it, it is possible to transform the resolutionby taking out a part of the DCT coefficient derived by thetwo-dimensional DCT to inversely transform by the DCT of a differentdegree, as shown in FIG. 59.

In a picture system such as a graphic display, in order to actualizevarious image effects, it is desired to perform the resolution transformof a content in a screen for the reduction and enlargement thereof.Since there are contents of various shapes, it is required to performthe resolution transform of contents of optional shapes. However, forexample, in the AS-DCT which is a method for performing the orthogonaltransform of optional shapes disclosed in the aforementioned Literature2, it is impossible to actualize the resolution transform in a casewhere a block to be transformed is an edge block, i.e. a blockcontaining the boundary portion of a content.

In addition, there are problems in that the coding efficiency to an edgeblock is low in the AS-DCT and other methods for performing theorthogonal transform of optional shapes.

SUMMARY OF THE INVENTION

The advantages and purpose of the invention are set forth in part in thedescription which follows, and in part are obvious from the description,or may be learned by practice of the invention. The advantages andpurpose of the invention are realized and attained by means of theelements and combinations particularly pointed out in the appendedclaims.

To attain the advantages and in accordance with the purpose of theinvention, as embodied and broadly described herein, the inventioncomprises an image data coding system. The system comprises of a firstcoding means for coding a map signal indicative of a position and shapeof a content on a display area of picture signals to be coded. Thesystem comprises a resolution transform means for transforming aresolution of the map signal. The system comprises an orthogonaltransform means for performing a first orthogonal transform and a twodimensional orthogonal transform, different from the first orthogonaltransform, of the picture signals in accordance with the resolutionafter transformed of the map signal to output an orthogonal transformcoefficient. The system comprises a second coding means for coding theorthogonal transform coefficient derived by the orthogonal transformmeans, wherein the orthogonal transform means performs thetwo-dimensional orthogonal transform of the picture signals of allpixels for blocks located inside a boundary portion of the content, andperforms the orthogonal transform of at least the picture signals of thepixels for blocks containing the boundary portion of the content.

An image data decoding system may comprise a first decoding means fordecoding a coded map signal indicative of a position and shape of acontent on a display area inputted for every block of picture signals.The system comprises a resolution transform means for performing aresolution transform of the map signal decoded by the first decodingmeans. The system comprises a second decoding means for decoding codedorthogonal transform coefficients. The system comprises a coefficientselecting means for selecting an orthogonal transform coefficientnecessary to reproduce an image of a predetermined resolution, from theorthogonal transform coefficients decoded by the second decoding means,based on the map signal resolution-transformed by the resolutiontransform means. The system comprises inverse orthogonal transform meansfor performing an inverse orthogonal transform of the orthogonaltransform coefficient selected by the coefficient selecting means. Thesystem comprises a reproducing means for deriving a regenerative picturesignal resolution-transformed from the results of the inverse orthogonaltransform by the inverse orthogonal transform means, wherein the inverseorthogonal transform means performs a two-dimensional inverse orthogonaltransform of all the coefficients for blocks located inside a boundaryportion of the content among the orthogonal transform coefficientsselected by the coefficient selecting means, and performs the inverseorthogonal transform of only the coefficients for blocks containing theboundary portion of the content.

An image data coding system may comprise a first coding means for codinga map signal indicative of a position and shape of a content on adisplay area of picture signals to be coded. The system comprises aresolution transform means for transforming a resolution of the mapsignal. The system comprises an average value separating means foroutputting values of pixels by outputting an average value of pixelswhen blocks include a boundary portion of the content, outputting adifference value between the average value and a pixel value when blocksare located inside the boundary portion of the content, and outputting azero value when blocks are located outside the boundary portion of thecontent. The system comprises an orthogonal transform means forperforming a two-dimensional orthogonal transform of the signals fromwhich the average value has been separated by the average valueseparating means, in order to output orthogonal transform coefficients.The system comprises a second coding means for coding the orthogonaltransform coefficients outputted by the orthogonal transform means andthe average value.

An image data decoding system may comprise a first decoding means fordecoding a coded map signal indicative of the position and shape of acontent on a display area inputted for every block of picture signals.The system comprises a resolution transform means for performing aresolution transform of the map signal decoded by the first decodingmeans. The system comprises a second decoding means for decoding codedorthogonal transform coefficients and an average value of pixels insideof the content. The system comprises a coefficient selecting means forselecting an orthogonal transform coefficient necessary to reproduce animage of a predetermined resolution, from the orthogonal transformcoefficients decoded by the second decoding means, based on the mapsignal resolution-transformed by the resolution transform means. Thesystem comprises an inverse orthogonal transform means for performing atwo-dimensional inverse orthogonal transform of the orthogonal transformcoefficient selected by the coefficient selecting means. The systemcomprises a reproducing means for deriving a resolution-transformedregenerative picture signal by synthesizing the results of the inverseorthogonal transform means with the average value decoded by the seconddecoding means, based on the map signal resolution-transformed by theresolution transform means.

An image data coding system may comprise a first coding means for codinga map signal indicative of a position and shape of a content on adisplay area of picture signals to be coded. The system comprises aresolution transform means for transforming a resolution of the mapsignal. The system comprises an average value inserting means forreplacing the values of pixels located outside a boundary portion of thecontent with an average value of the values signal of pixels locatedinside the boundary portion of content, in accordance with theresolution after transformed of the map signal. The system comprises anorthogonal transform means for performing a two-dimensional orthogonaltransform of the average value signal in the blocks produced by theaverage value inserting means, in order to output orthogonal transformcoefficients. The system comprises a second coding means for coding theorthogonal transform coefficients outputted by the orthogonal transformmeans.

An image data decoding system may comprise a first decoding means fordecoding a coded map signal indicative of a position and shape of acontent on a display area inputted for every square block of picturesignals. The system comprises a resolution transform means forperforming a resolution transform of the map signal decoded by the firstdecoding means. The system comprises a second decoding means fordecoding coded orthogonal transform coefficients. The system comprises acoefficient selecting means for selecting an orthogonal transformcoefficient necessary to reproduce an image of a predeterminedresolution, from the orthogonal transform coefficients decoded by thesecond decoding means, based on the map signal resolution-transformed bythe resolution transform means, an inverse orthogonal transform meansfor performing a two-dimensional inverse orthogonal transform of theorthogonal transform coefficient selected by the coefficient selectingmeans. The system comprises a reproducing means for deriving aresolution-transformed regenerative picture signal by taking out thevalues of pixels located inside a boundary portion of the content fromthe results of the two-dimensional inverse orthogonal transform by theinverse orthogonal transform means, based on the map signalresolution-transformed by the resolution transform means.

An image data coding system may comprise a first coding means for codinga map signal indicative of a position and shape of a content on adisplay area of picture signals to be coded. The system comprises aresolution transform means for transforming a map signal. The systemcomprises a vector quantizing means for quantizing the picture signalsby matching the picture signals with code vectors stored in a code bookso as to output an index indicative of a code vector having the highestcorrelation to the picture signals when blocks are located inside aboundary portion of the content, and matching at least the picturesignals inside the boundary portion of the content with the code vectorswhen blocks contain the boundary portion of the content in accordancewith the resolution after transformed of the map signal. The systemcomprises a second coding means for coding the index outputted by thevector quantizing means.

An image data decoding system may also comprise a first decoding meansfor decoding a coded map signal indicative of a position and shape of acontent on a display area inputted for every block of picture signals tobe coded. The system comprises a resolution transform means forperforming a resolution transform of the map signal decoded by the firstdecoding means. The system comprises a second decoding means fordecoding a coded index. The system comprises an inverse vectorquantizing means, having a code book that stores code vectors indicatedby multiple resolutions, for outputting a code vector of a desiredresolution designated by the index decoded by the second decoding meanswhen blocks are located inside a boundary portion the content, andderiving a resolution-transformed regenerative picture signal by takingout only the signal located inside the boundary portion of the content,when the blocks contain the boundary portion of the content, from thecode vectors according to the map signal resolution-transformed by theresolution transform means.

An image data coding system may also comprise a first coding means forcoding a map signal indicative of a position and shape of a content on adisplay area of picture signals to be coded. The system comprises asubband dividing means for dividing the picture signal into a pluralityof subband picture signals. The system comprises a resolution transformmeans for performing a resolution transform of the map signal into theresolution of each of the subband picture signals divided by the subbanddividing means. The system comprises a second coding means for codingeach of the subband picture signals divided by the subband dividingmeans when blocks are located inside a boundary portion of the content,and coding at least the signals located inside the boundary portions ofthe content when the blocks contain the boundary portion of the contentin the subband picture signals according to the map signalresolution-transformed by the resolution transform means.

An image data decoding system may also comprise a first decoding meansfor decoding a coded map signal indicative of a position and shape of acontent on a display area of picture signals to be coded. The systemcomprises a resolution transform means for performing a resolutiontransform of the map signal decoded by the first decoding means, intothe resolutions of a plurality of subband picture signals. The systemcomprises a second decoding means for decoding a plurality of codedsubband signals in accordance with the map signal resolution transformedby the resolution transform means. The system comprises a subbandsynthesizing means for deriving a resolution-transformed regenerativepicture signal by synthesizing at least the subband picture signalsnecessary to reproduce an image of a predetermined resolution among theplurality of subband picture signals decoded by the second decodingmeans.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood more fully from the detaileddescription given hereafter and from the accompanying drawings of thepreferred embodiments of the invention. However, the drawings are notintended to imply limitation of the invention of a specific embodiment,but are for explanation and understanding only.

In the drawings:

FIG. 1 is a block diagram showing a basic concept of an image datacoding system, according to the present invention;

FIG. 2 is a block diagram showing the detail structure of a coding meansof an image data coding system, according to the present invention;

FIG. 3 is a schematic block diagram of the first preferred embodiment ofan image data coding system, according to the present invention;

FIG. 4 is a view explaining the state of the distribution of closelyobserved points in the first preferred embodiment of an image datacoding system, according to the present invention;

FIGS. 5(a) and 5(b) are views explaining the control of the coding inaccordance with the distribution of regions in the first preferredembodiment of an image data coding system, according to the presentinvention;

FIG. 6 is a view explaining the control of the coding of coefficient bythe subband coding in the first preferred embodiment of an image datacoding system, according to the present invention;

FIGS. 7(a) and 7(b) are views explaining the state that the quantizationcharacteristic is changed by changing the dead zone of a quantizer inthe first preferred embodiment of an image data coding system, accordingto the present invention;

FIG. 8 is a schematic block diagram of the second preferred embodimentof an image data coding system, according to the present invention;

FIG. 9 is a detailed block diagram of a space-time filter in the secondpreferred embodiment of an image data coding system, according to thepresent invention;

FIGS. 10(a) and 10(b) are views showing pixels being objects of thespace filter processing by the space-time filter in the second preferredembodiment of an image data coding system, according to the presentinvention;

FIG. 11 is a schematic block diagram of the third preferred embodimentof an image data coding system, according to the present invention;

FIG. 12 is a schematic block diagram of the fourth preferred embodimentof an image data coding system, according to the present invention;

FIGS. 13(a) to 13(c) are views explaining the processing of assignmentof the code amount in the fourth preferred embodiment of an image datacoding system, according to the present invention;

FIG. 14 is a schematic block diagram of the fifth preferred embodimentof an image data coding system, according to the present invention;

FIG. 15 is a schematic block diagram of the sixth preferred embodimentof an image data coding system, according to the present invention;

FIG. 16 is a schematic block diagram of the seventh preferred embodimentof an image data coding system, according to the present invention;

FIG. 17 is a schematic block diagram of the eighth preferred embodimentof an image data coding system, according to the present invention;

FIG. 18 is a schematic block diagram of the ninth preferred embodimentof an image data coding system, according to the present invention;

FIG. 19 is a block diagram of the tenth preferred embodiment of an imagedata coding system, according to the present invention;

FIG. 20 is block diagram of an image data coding system in the firstpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 21 is a block diagram of an orthogonal transform circuit in FIG.20;

FIG. 22 is a block diagram of an inverse orthogonal transform in FIG.20;

FIG. 23 is a block diagram of an AS-DCT circuit in FIG. 21;

FIG. 24 is a block diagram of an AS-IDCT circuit in FIG. 21;

FIG. 25 is a view showing a transform method for the AS-DCT;

FIG. 26 is a block diagram of an image data decoding system in the firstpreferred embodiment of an image data coding and/or decoding system,according to the present invention ;

FIG. 27 is a view showing a resolution transform method in the firstpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 28 is a block diagram of an image data coding system in the secondpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 29 is a block diagram of an orthogonal transform circuit in FIG.28;

FIG. 30 is a block diagram of an inverse orthogonal transform circuit inFIG. 28;

FIG. 31 is a block diagram of an AS-DCT circuit in FIG. 29;

FIG. 32 is a block diagram of an AS-IDCT circuit in FIG. 30;

FIG. 33(a) and 33(b) are view showing a switching operation of atransforming order for AS-DCT in the second preferred embodiment of animage data coding and/or decoding system, according to the presentinvention;

FIG. 34 is a block diagram of an image data decoding system in thesecond preferred embodiment of an image data coding and/or decodingsystem, according to the present invention;

FIG. 35 is a view showing an example of a method for determining a scanorder in the first and second preferred embodiment of an image datacoding and/or decoding system, according to the present invention;

FIG. 36 is a block diagram of a scan order determining circuit foractualizing a scalable function in the AS-DCT;

FIG. 37 is a block diagram of an image data coding system in the thirdpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 38 is a view showing a method for separating an average value inthe image data coding system in FIG. 37;

FIG. 39 is a block diagram of an image data decoding system in the thirdpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 40 is a view showing a method for synthesizing an average value inthe image data decoding system of FIG. 39;

FIG. 41 is a block diagram of an image data coding system in the fourthpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 42 is a view showing a method for inserting an average value in theimage data coding system of FIG. 41;

FIG. 43 is a block diagram of an image data decoding system in thefourth preferred embodiment of an image data coding and/or decodingsystem, according to the present invention;

FIG. 44 is a view showing a method for separation of pixels in the imagedata decoding system of FIG. 43;

FIG. 45 is a block diagram of an image data coding system in the fourthpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 46 is a view explaining a method for the vector quantization of ablock of an optional shape in a vector quantizer of FIG. 45;

FIG. 47 is a view explaining a method for the inverse vectorquantization of a block of an optional shape in an inverse quantizer ofFIG. 45;

FIG. 48 is a block diagram of an image data decoding system in the forthpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 49 is a view showing a code block provided in the inverse vectorquantizer of FIG. 48;

FIG. 50 is a view explaining the subband division of a picture signal inthe fifth preferred embodiment of an image data coding and/or decodingsystem, according to the present invention;

FIG. 51 is a view showing the arrangement of the respective componentson the axes when a picture signal is divided into four subbands in thefifth preferred embodiment;

FIG. 52 is a block diagram of an image data coding system in the fifthpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 53 is a block diagram of an image data decoding system in the fifthpreferred embodiment of an image data coding and/or decoding system,according to the present invention;

FIG. 54 is a view showing an example of an image transmitting system towhich an image data coding system and an image data decoding system,according to the present invention, are applied;

FIG. 55 is a view explaining the principle of a conventional image datacoding system;

FIG. 56 is a view explaining a method for separating a picture signalinto a background and a content for coding;

FIG. 57 is a view explaining a conventional content-based coding;

FIGS. 58(a) and 58(b) are views explaining a conventional method of theorthogonal transform of an optional shape; and

FIG. 59 is a view explaining a method for actualizing the resolutiontransform using the orthogonal transform.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring now to the accompanying drawings, the preferred embodiments ofthe present invention will be described in detail below.

It is an object of the present invention to provide an image data codingsystem which can efficiently compress information by changing only theassignment of the code amount without changing the absolute amountthereof in accordance with the decreasing of the size of a reproducedpicture display, in view of the relationship between the size (area) ofthe screen and the distributing area of closely observed points.

An image data coding system, according to the present invention,comprises a screen area determining means for determining the size(area) of a screen reproduced on the basis of inputted image datasignals, a code amount assigning control means for controlling theassignment of the code amount of data for every region on the screen onthe basis of the results of determination and a coding means for codingthe image data signals inputted in accordance with the code amountassigned to every region.

The weight function for assigning the code amount corresponding to thesize of the screen is set so as to be changed by internally analyzingthe inputted image data signals to determine the size of the screen, orby designating the size of the screen in an externally manual set mode.The assigned amount of the code amount is determined using the setweight function, and the coding of the image data signal is performed onthe basis of the assigned amount. Therefore, the assignment of the codeamount is changed using the weight function in accordance with the areaof the screen, so that it is possible to provide the optimum screen forpractical use by only determining or designating the size of the screenif the weight function is set in view of human's visual characteristic.

In an image data coding system, according to the present invention, thescreen-area determining means may internally determine the area of thescreen of the produced picture on the basis of the amount of theinputted image data signals and so forth, or externally designating thesize of the screen in a manual operation. In the case of thedetermination by internal processing, the amount of the image datasignals may be detected to detect the resolution of the produced screenon the basis of the number of pixels of the produced screen.Alternatively, the information relating to the size of the screen, whichinformation are included in a part of the image data signals, may betransmitted to analyze the information by a determining means todetermine the size of the screen.

In addition, it is an object of the present invention to provide animage data coding and/or decoding system which can perform theresolution transform of the blocks containing the boundary portion of acontent.

It is also an object of the present invention to provide an image datacoding and/or decoding system capable of high-efficient coding of theblocks containing the boundary portion of a content.

With respect to the pixels in the blocks containing the boundary portionof the content, the orthogonal transform means may perform theone-dimensional orthogonal transform in the horizontal or verticaldirection after rearranging the pixels contained in the content in thehorizontal or vertical direction, and performs the one-dimensionalorthogonal transform in the vertical or horizontal direction afterputting the derived transform coefficients in order of the lower band ofcoefficient.

In this case, it may be provided with a correlation detecting means fordetecting the respective correlations in the horizontal and verticaldirections of the picture signals inside of the content, to switch thedirection of the one-dimensional orthogonal transform so as to performthe one-dimensional orthogonal transform in order of the direction thatthe correlation is higher.

With respect to the blocks containing the boundary portion of thecontent, the inverse orthogonal transform in the horizontal or verticaldirection after rearranging the transform coefficients contained in thecontent in the horizontal or vertical direction, and performs theone-dimensional inverse orthogonal transform in the vertical orhorizontal direction after rearranging them to the former positions ofpixels.

When the first image data coding system switches the direction of theone-dimensional orthogonal transform so as to perform theone-dimensional orthogonal transform in order of the directiondetermined that the correlation is higher in the horizontal and verticaldirections of the picture signals inside of the content, the first imagedata decoding system may switch the direction of the one-dimensionalinverse orthogonal transform on the basis of the switching informationof the first image data coding system.

In the average value inserting means, the values of pixels outside ofthe content may be predicted under the condition that the average valueof the pixels outside of the content coincides with the average value ofthe pixels inside of the content.

In an image data coding and/or decoding system, according to the oneaspect of the present invention, it is possible to code transformcoefficients and a map signal in a coding system, by performing thetwo-dimensional orthogonal transform of the picture signals of all thepixels with respect to the blocks (inside blocks) located inside of acontent, and of only the picture signals of pixels contained in thecontent with respect to the blocks (edge blocks) containing the boundaryportion of the content, in accordance with a map signal indicative ofthe position and shape of the content. It is also possible to performthe resolution transform with respect to the edge blocks containing acontent of an optional shape in a decoding system, by selecting anorthogonal transform coefficient necessary to reproduce an image of adesired resolution from decoded orthogonal transform coefficients on thebasis of a decoded and resolution-transformed map signal, and byperforming the two-dimensional orthogonal transform of all thecoefficients with respect to the inside blocks and of only thecoefficients contained in the content with respect to the edge blocks,respectively.

In this case, it is designed to be able to switch the order of theone-dimensional orthogonal transform in the horizontal and verticaldirections in the two-dimensional orthogonal transform, to detect thecorrelation in the horizontal and vertical directions of the picturesignals inside of the content, for performing, first, theone-dimensional orthogonal transform with respect to the directionhaving higher correlation, so that it is possible to improve the codingefficiency.

In an image data coding and/or decoding system, according to anotheraspect of the present invention, in a coding system, by coding a mapsignal, outputting an average value of the values of pixels inside of acontent with respect to blocks containing the boundary portion of thecontent among picture signals in accordance with the map signal,separating the average value from the values of pixels inside of thecontent, and setting the values of pixels outside of the content to bezero, to perform the two-dimensional orthogonal transform of the signalsfrom which the average value has been separated, it is possible to codethe orthogonal transform coefficients and the average value. In adecoding system, by selecting an orthogonal transform coefficientnecessary to reproduce an image of a desired resolution from decodedorthogonal transform coefficients on the basis of a decoded andresolution-transformed map signal, and deriving a resolution-transformedregenerative picture signal by synthesizing the results of thetwo-dimensional inverse orthogonal transform with the decoded averagevalue of the values of pixels inside of the content, it is possible toperform the resolution transform with respect to the edge blockscontaining a content of an optional shape. In addition, it is possibleto enhance the coding efficiency in the edge blocks by separating theaverage value inside of the content from the average value outsidethereof for coding.

In an image data coding and/or decoding system, according to anotheraspect of the present invention, a coding system can code a map signal,replace the values of pixels outside of a content by the average valueof the values of pixels inside of the content with respect to blockscontaining the boundary portion of the content among the picture signalsin accordance with a map signal, output the replaced values, and performthe two-dimensional orthogonal transform of the signal of the averagevalue in the block to code its orthogonal transform coefficient. Inaddition, a decoding system can select an orthogonal transformcoefficient necessary to reproduce an image of a predeterminedresolution from coded orthogonal transform coefficients on the basis ofa coded and resolution-transformed map signal, and take out the valuesof pixels inside of the content with respect to the edge blocks on thebasis of the results of the two-dimensional orthogonal transform toderive a resolution-transformed regenerative picture signal, so that itis possible to perform the resolution transform with respect to the edgeblocks containing a content of an optional shape.

In an image data coding and/or decoding system, according to anotheraspect of the present invention, a coding system can code a map signal,perform the matching only the signals inside of a content with a codevector with respect to the edge blocks in accordance with the mapsignal, perform the vector quantization, and code an index indicative ofthe code vector of the highest correlation. In addition, in a decodingsystem, when performing the inverse vector quantization of the codevector designated by the decoded index, only the signal inside of thecontent are taken out from the code vector with respect to the edgeblocks in accordance with a decoded and resolution-transformed mapsignal, to derive a resolution-transformed regenerative picture signal,so that it is possible to perform the resolution transform with respectto the edge blocks containing a content of an optional shape.

In an image data coding and/or decoding system, according to anotheraspect of the present invention, when picture signals are divided intosubbands to be coded in a coding system. only the signals inside of acontent with respect to the edge blocks in subband picture are coded inaccordance with a map signal resolution-transformed into resolutions ofsubband picture signals. In addition, in a decoding system, whensubband-synthesizing only the subband picture signal necessary to derivea regenerative picture signal of a predetermined resolution, only thesignals inside of the content with respect to the edge blocks among thesubband picture signals are decoded in accordance with a map signalresolution-transformed into the resolution of each of the subbandpicture signals, so that it is possible to perform the resolutiontransform with respect to the edge blocks containing a content of anoptional shape.

According to the present invention, a method for performing thetwo-dimensional orthogonal transform and/or the inverse orthogonaltransform for blocks of an optional shape is provided. That is, atwo-dimensional orthogonal transform method, according to the presentinvention, comprises a first transform step for performing theone-dimensional orthogonal transform in the horizontal direction inaccordance with a map signal indicative of the shape of a blockinputted, and for performing the rearrangement in order of the lower ofcoefficients in the horizontal direction, and a second transform stepfor performing the one-dimensional orthogonal transform in the verticaldirection in accordance with the map signal, and for performing therearrangement in order of the lower of coefficients in the verticaldirection, wherein with respect to a signal of an optional shape, thesecond transform step is performed after performing the first transformstep, or the first transform step is performed after performing thesecond transform step.

According to the present invention, a two-dimensional inverse orthogonaltransform method adapted to the aforementioned two-dimensionalorthogonal transform method, comprises a resolution transform step forperforming the resolution transform of an input map signal; acoefficient selecting step for selecting an orthogonal transformcoefficient necessary to reproduce an image of the resolution inaccordance with a resolution-transformed map signal; a first inversetransform step for performing the one-dimensional orthogonal transformin the vertical direction with respect to the selected orthogonaltransform coefficient, and for performing the rearrangement in thevertical direction; and a second inverse transform step for performingthe one-dimensional orthogonal transform in the horizontal directionwith respect to the selected orthogonal transform coefficient, and forperforming the rearrangement in the horizontal direction, wherein aresolution-transformed signal of a block of an optional shape isreproduced by performing the first inverse transform step prior to thesecond inverse transform step when the first transform step is performedprior to the second transform step, and by performing the second inversetransform step prior to the first inverse transform step when the secondtransform step is performed prior to the first transform step.

FIG. 1 is a block diagram showing the basic concept of the presentinvention. In this drawing, the reference numeral 1 denotes an inputterminal for inputting an image data signal, 3 denotes an screen-areadetermining means for determining the area of the produced screen byanalyzing the image data signal S1 inputted through the terminal 1 or byexternally manual operations, 4 denotes a code amount assigning controlmeans for outputting a control signal S2 which controls the assignmentof the code amount using a weight function corresponding to the area ofthe screen on the basis of the determined results from the screen-areadetermining means 3, and 10 denotes a coding means for coding the imagedata signal S1 inputted using the control signal S2 from the code amountassigning control means.

FIG. 2 is a detailed block diagram of the coding means 10 of FIG. 1. Theimage data signal S1 is supplied to a motion-vector detection circuit11, a differential circuit 12 and a mode determining circuit 13. Thecoding means 10 further comprises: a reference frame memory 14, havingvariable delay function for motion-compensation, for storing a referenceframe; a selector 15 for outputting an input signal or a differentialsignal; a selector 16 for outputting a signal of zero level or amotion-compensation signal; a DCT circuit 17 for performing the discretecosine transform (DCT) of the output from the selector 15; and aquantization and inverse quantization means 20 for performing thequantization and inverse quantization processing of the output from theDCT circuit 17 in accordance with the weighting by the inputted controlsignal S2; an inverse DCT circuit 18 for performing the inverse discretecosine transform of the output from the quantization and inversequantization means 20; and an addition circuit 19 for adding the outputsfrom the selector 16 and the inverse quantization circuit 18. Thequantization and inverse quantization means 20 comprises a quantizationcircuit 21 and an inverse quantization circuit 22.

With this construction, the motion-vector detecting circuit 11 detects amotion-vector between the reference frame stored in the reference framememory 14 having the variable delay function for motion-compensation,and the inputted image data signal S1, for every microblock (MB)composed of 16×16 pixels.

The differential circuit 12 derives a difference between themotion-compensation signal S3 of the reference frame outputted from thereference frame memory 14, and the inputted image data signal S1, tosupply the difference to the mode determining circuit 13 and theselector 15.

The mode determining circuit 13 compares the differential signal S4outputted from the differential circuit 12 with the AC component for theinputted image data signal, to determine as to whether the intraframe orinterframe coding of the block is performed. The determined results aresupplied to the selector 15 and the selector 16 via the selector 15.

The selector 15 selects the inputted image data signal S1 when it isdetermined to perform the intraframe coding, and the differential signalS4 when it is determined to perform the interframe coding, to supply aselected signal S5 to the DCT circuit 17.

The DCT circuit 17 transforms the selected signal S5 into the discretecosine transform coefficient S6, to supply it for the quantization andinverse quantization means 20.

The quantization and inverse quantization means 20 quantizes thediscrete cosine transform coefficient S6 supplied from the DCT circuit17, in accordance with the control signal S2 relating to thequantization step size supplied for the quantization circuit 21 from arate control circuit (not shown) of the code amount assigning controlmeans, to output a transform coefficient signal S7. This quantizationsignal S7 is also supplied to the inverse quantization circuit 22, sothat the inverse quantization circuit 22 performs the inversequantization of the transform coefficient signal S7 into the discretecosine transform coefficient S6 in accordance with the control signal S2relating to the quantization step size.

The inverse DCT circuit 18 performs the discrete cosine inversetransforms of the discrete cosine transform coefficient S6 formed by theinverse quantization, to reproduce any signals selected by the selector15. That is, when it is determined to perform the intraframe coding, asignal corresponding to the image data signal S1 is reproduced, and whenit is determined to perform the interframe coding, a signalcorresponding to the differential signal S4 is reproduced. The signalproduced by the inverse transform by the inverse DCT transform circuit18 is supplied to the addition circuit 19.

On the other hand, the selector 16 selects the signal of zero level whenthe mode determined by the mode determining circuit 13 is the intraframecoding, and it selects the motion-compensation estimating signal S3stored in the reference frame memory 14 when the mode is the interframecoding, to supply the selected signal to the addition circuit 19. Theaddition circuit 19 adds the output of the selector 16 to the output ofthe inverse DCT circuit 18, to supply it to the reference frame memory14. The reference frame memory 14 stores therein the addition signaloutputted from the addition circuit 19, and supplies the reference framesignal when the motion-vector detection circuit 11 performs themotion-vector detection.

Furthermore, the transform coefficient S7 quantized by the quantizationcircuit 21 is variable-length coded with side information such asmotion-vector, and then, it is multiplexed to be outputted.

Referring to FIGS. 3 to 7, the first preferred embodiment of an imagedata coding system, according to the present invention, will bedescribed below.

In FIG. 3, an encoder 25 for coding the input image data S1 is providedbetween the input terminal 1 and the output terminal 2. The image datasignal S1 is also supplied to the resolution detecting circuit 23serving as the screen-area determining means 3. The resolution detectingcircuit 23 detects the resolution (the number of pixels) of the inputimage data signal, to supply information on the number of pixels to thecode-amount assigning control circuit 24.

The code-amount assigning control circuit 24 first varies the weightdistribution function for assigning the code amount corresponding to theposition in the screen in accordance with the number of the pixels inthe input image. The weight distribution function for the assignment ofthe code amount may be the standard deviation of a two-dimensionalnormal distribution as a function of the number of pixels, as shown inFIG. 5.22 of the aforementioned literature "Estimation Technique ofImage Quality and Tone Quality". FIG. 4 is a rewritten view of FIG. 5.22of the aforementioned literature, and shows the relationship between thedistribution of closely observed points 27 in a contour 26 of a screenin a current television system, and the distribution of closely observedpoints 29 in a contour 28 of a screen in a high quality televisionsystem. As can be clearly seen from this drawing, the distribution ofclosely observed points extends as the size of the screen increases.Therefore, the two-dimensional normal distribution of closely observedpoints can be applied to the weight distribution function as a functionof the number of pixels.

As shown in FIGS. 5(a) and 5(b), the weight distribution function forthe assignment of the code amount may switch the weight for every regiondivided into microblocks (MB). That is, in FIG. 5(a), the blocks dividedby dotted lines are microblocks (MB), the weighting is performed so thatthe produced coded-amount of the region 31 is less than that of theregion 32. This distribution function is set so that the weight of thecentral portion of the screen for the image having a less number ofpixels as shown in FIG. 5(a) is greater than that for the image having amore number of pixels as shown in FIG. 5(b). The weight distributionfunction thus set is supplied to the encoder 25.

In accordance with the distribution function for the assignment of thecode amount supplied by the code-amount assigning control circuit 24,the encoder 25 performs the weighting of the code amount produced byvarying the quantization characteristic in accordance with the positionof the pixel or the position of the block on the screen in thequantization and inverse quantization circuit 20.

As a first method for varying the quantization characteristic, in thecoding using the orthogonal transform and the subband coding, withrespect to the first regions 31 and 33 located on the periphery of FIGS.5(a) and 5(b), the coefficient of a higher frequency component than thatof the boundary b1 of FIG. 6 is not compulsorily coded, and with respectto the second regions 32 and 34 located at an intermediate portion, thecoefficient of a higher frequency component than that of the boundary b2is not compulsorily coded.

In a second method for varying the quantization characteristic, thequantization matrix weighted for every transform coefficient is switchedbetween the first and second regions 31 and 32 of FIG. 5(a) or betweenthe first and second regions 33 and 34 of FIG. 5(b).

In a third method for varying the quantization characteristic, as shownin FIG. 7, the dead zone of the quantizer is changed. In FIG. 7, thereference numeral 35 denotes a dead zone, and 36 denotes typical valuesof quantization.

Referring to FIGS. 8 to 10, the second preferred embodiment of an imagedata coding system, according to the present invention, will bedescribed in detail below.

The image data signal S1 is supplied to the frame memory 38. This framememory 38 supplies an image signal S8 of the current frame to aresolution detecting circuit 23 and a space-time filter 40, and an imagesignal S9 of the previous frame only to the space-time filter 40.

FIG. 9 shows the detailed structure of the space-time filter 40. Asshown in FIG. 9, the space-time filter 40 comprises an intraframe filtercircuit 41 for performing the space filter processing of the imagesignal S8 of the current frame, a multiplication circuit 42 formultiplying the output of the filter circuit 42 in the frame by k, amultiplication circuit 43 for multiplying the image signal S9 of theinputted previous frame by "1-k", and an addition circuit 44 for addingthe multiplied outputs of the multiplication circuits 42 and 43.

In a case where the output ⋆X of a pixel X shown in each of FIGS. 10(a)and 10(b) is derived from the intraframe filter circuit 41 of thespace-time filter 40, an example of operation formula is as follows.

    ⋆X=(A+mB+C+mD+mE+F+mG+H+m2X)/(m+2)2

wherein m is a variable for varying strength of the space filter.

The output ⋆X of the intraframe space filter circuit 41 is multiplied byk by means of the multiplication circuit 42, to be added in the additioncircuit 44 to the value derived by multiplying the input image signal Pof the previous frame by "1-k" by means of the multiplication circuit42, so that the time filter processing is carried out. Theaforementioned k is a coefficient for varying the strength of the timefilter. Furthermore, FIG. 10(a) is a view showing the relationshipbetween the positions of the pixel X and P. The coefficients m and k areset so that the value of coefficient in the first region 31 of FIG. 5(a)is less than that in the second region 32 thereof, in accordance withthe code-amount weight distribution function contained in the controlsignal S2 supplied from the code-amount assigning control circuit 24. Inthis way, the space-time filtering to the image signal of the firstregion 31 is strengthly performed, so that it is possible to restrainthe produced code amount. The output of the space-time filter 40 issupplied to the encoder 25 as a signal S10, and coded here to beoutputted to the outside via the output terminal 2.

Referring to FIG. 11, the third preferred embodiment of an image datacoding system, according to the present invention will be describedbelow.

The third preferred embodiment of an image data coding system isdifferent from the second preferred embodiment of the system as shown inFIG. 8, at the points that the encoder 25 in the second preferredembodiment of the image data coding system comprises the samequantization and inverse quantization circuit 20 as that in the firstpreferred embodiment of the image data coding system as shown in FIG. 3,and that the code-amount assigning weight distribution function is notonly supplied to the space-time filter 40, but also supplied to thequantization and inverse quantization circuit 20 of the encoder 25.

In FIG. 11, the coefficients m and k of the space-time 40 are set sothat the value of coefficient of the first region 31 of FIG. 5 is lessthan that of the second region 32 thereof, in accordance with thecode-amount assigning weight distribution function contained in thecontrol signal S2 supplied from the code-amount assigning controlcircuit 24. In this way, the space-time filter processing of the firstregion 31 of FIG. 5 is stronger than that of the second region 32, sothat the code amount produced in the first region 31 can be restrained.

The output of the space-time filter 40 is supplied to the encoder 25 asa signal S10, and coded to be outputted. In this encoder 25, since thecode-amount assigning weight distribution function contained in thecontrol signal S2 supplied from the code-amount assigning controlcircuit 24 is also supplied to the quantization and inverse quantizationcircuit 20, the quantization and inverse quantization circuit 20performs the weighting of the produced code amount so as to vary thequantization characteristic in accordance with the positions of thepixels and the blocks on the screen, in the same manner as that of thefirst preferred embodiment. The image signal weighted so as to vary thequantization characteristic by the positions on the screen is coded withthe produced code amount which is different in accordance with thepositions, and then, outputted to the outside via the output terminal 2.

Referring to FIG. 12, the fourth preferred embodiment of an image datacoding system, according to the present invention, will be describedbelow.

This fourth preferred embodiment of an image data coding system is notprovided with the space-time filter 40 in the third preferred embodimentof an image data coding system. In this embodiment, the image signal ofthe current frame of the frame memory 38 is supplied to the encoder 25,and the respective image signals of the current and previous frames aresupplied to a face-region detecting circuit 45, so that the quantizationcharacteristic of the quantization and inverse quantization circuit 20is varied by a control signal S2 containing the code-amount assigningweight distribution function outputted from the code-amount assigningcontrol circuit 24 which receives the output of the face-regiondetecting circuit 45 and the output of the resolution detecting circuit23.

With the aforementioned construction, the face-region detecting circuit45 detects the face region in the same manner as that of "Image DataCoding System" disclosed in the aforementioned Japanese Patent First(unexamined) Publication No. 5-95541, and the detected results aresupplied to the code-amount assigning control circuit 24 as an outputsignal S11.

In the code-amount assigning control circuit 24, first, as shown in FIG.13(a), the first region 31 and the second region 32 are determined inaccordance with the number of pixels of the input image in the samemanner as that of the first preferred embodiment, to vary the weightfunction for the assignment of the code amount in accordance with thepositions on the screen. Then, the face region 47 of FIG. 13(b) isdetected, and the weight distribution function is modified as shown inFIG. 13(c) in view of the detected results of the face region 47. Anexample of a method for this modification is as follows.

The interior of the face region 47 of FIG. 13(c) is assumed to be athird region 53. A part of the second region 32 of FIG. 13(a) which isnot contained in the face region 47 of FIG. 13(c) becomes the secondregion 32. If a portion contained in the first region 31 of FIG. 13(1)is contained in the face region, this portion becomes the first region31. A portion contained in the second region 32 and contained in theface region 47 of FIGS. 13(b) and 13(c) serves as the third region 53 ofFIG. 13(c).

The weight distribution function as shown in FIG. 13(c) is supplied tothe encoder 25.

In the encoder 25, in accordance with the code-amount assigning weightdistribution function, the quantization and inverse quantization circuit20 performs the weighting of the produced code amount by varying thequantization characteristic in accordance with the positions of thepixel and the block on the screen, in the same manner as that of thefirst preferred embodiment.

Referring to FIG. 14, the fifth preferred embodiment of an image datacoding system, according to the present invention, will be describedbelow.

This fifth preferred embodiment of an image data coding system comprisesthe combination of the second preferred embodiment of the system asshown in FIG. 8 with the fourth preferred embodiment of the system asshown in FIG. 12.

In FIG. 14, the image signal S8 of the current frame outputted from theframe memory 38 is supplied to three circuits, i.e. the resolutiondetecting circuit 23, the space-time filter 40 and the face regiondetecting circuit 45. The image signal S9 of the last frame is suppliedto both of the space-time filter 40 and the face region detectingcircuit 45. The outputs of the resolution detecting circuit 23 and theface region detecting circuit 45 are supplied to the code-amountassigning control circuit 24, so that the code-amount assigning weightdistribution function is set. On the basis of this weight distributionfunction, the space-time filter 40 performs the space-time filterprocessing for the image signals S8 and S9 of the current and previousframes, and outputs a signal S10 to the encoder 25. The encoder 25 codesthis signal S10 to output to the outside via the output terminal 2.

FIG. 15 is a block diagram showing the sixth preferred embodiment of animage data coding system, according to the present invention. In thissixth preferred embodiment, the encoder 25 in the fifth preferredembodiment corresponds to one comprising the quantization and inversequantization circuit 20 in the second preferred embodiment. Since othercomponents are the same as or correspond to the components having thesame reference numerals as those in some preferred embodiments asmentioned above, only such reference numerals are used in the drawing,and the repeated explanations are omitted.

The space-time filter 40 of FIG. 14 receives a control signal S2containing the code-amount assigning weight distribution functionsupplied for the code-amount assigning control circuit 24, and performsthe space-time filter processing to output a signal S10 to the encoder25.

In the encoder 25, in the weight distribution function contained in thecontrol signal supplied from the code-amount assigning control circuit24 in the same manner as that of the first preferred embodiment, thequantization and inverse quantization circuit 20 varies the quantizationcharacteristic in accordance with the positions of the pixel and theblock on the screen, to perform the weighting of the produced codeamount.

FIG. 16 is a block diagram showing the seventh preferred embodiment ofan image data coding system, according to the present invention. In FIG.16, the characterizing feature of this seventh preferred embodiment isthat a sink screen-size detecting circuit 50 is provided for receivingan output signal from the frame memory 38 to detect the size of the sinkscreen, and that the code-amount assigning control circuit 24 derivesthe code-amount assigning weight distribution function on the basis ofboth of the output signals of the sink screen-size detecting circuit 50and the frame memory 38, and the derived function is supplied to thequantization and inverse quantization circuit 20 of the encoder 25. In acase where the space-time filter is provided, the same operation as thatof the preferred embodiment of an image data coding system is performed,so that the repeated explanations are omitted.

FIG. 17 is a block diagram showing the eighth preferred embodiment of animage data coding system, according to the present invention.

In this drawing, the same face-region detecting circuit 45 as that inthe fourth preferred embodiment as shown in FIG. 12 is provided inaddition to the structures in the seventh preferred embodiment as shownin FIG. 16. To the code-amount assigning control circuit 24, the outputof the face-region detecting circuit 45 in addition to the output of thesink screen-size detecting circuit 50 are supplied. Therefore, on thebasis of the output of the sink screen-size detecting circuit 50 and theoutput of the face-region detecting circuit 45, the code-amountassigning control circuit 24 sets the code-amount assigning weightdistribution function by the image data inputted through the framememory 38, to output it to the quantization and inverse quantizationcircuit 20 of the encoder 25. The quantization and inverse quantizationcircuit 20 varies the quantization characteristic on the basis of thesupplied distribution function, and performs the weighting of theproduced code amount to output a signal to the outside via the terminal2.

In the seventh and eighth preferred embodiments of an image date codingsystem, according to the present invention, it is possible to easilydetect, on the sink, the size of the screen of the received informationto be reproduced, the information being transmitted by transmitting theheader information indicative of the size of the screen and so forth inaddition to the image data signal.

In the seventh and eighth preferred embodiments of an image data codingsystem as shown in FIGS. 16 and 17, the size of the screen is internallyand automatically detected on the sink to be controlled, by the sinkscreen-size detecting circuit 50 serving as means for detecting the sizeof the screen supplied to the code-amount assigning control circuit 24.However, the present invention is not limit to this structure, but theweighting of the produced code amount may be performed by input in anexternally manual operation.

That is, the ninth and tenth preferred embodiments of image data codingsystems, as shown in FIGS. 18 and 19, according to the presentinvention, may be applied.

FIG. 18 is a schematic view of the ninth preferred embodiment of animage data coding system, according to the present invention. This ninthpreferred embodiment of an image data coding system does not detect thesize of the screen of the received information in the sink screen-sizedetecting circuit 50 in the seventh preferred embodiment of the imagedata coding system, and a screen-size setting means 55 is provided forsetting the size of the screen in an externally manual operation. Thescreen-size setting means 55 does not detect the resolution of thereceived image data signal, the header information indicative of thearea and so forth, to derive the size of the screen, but is designed soas to input the size of the received screen to the sink system in amanual operation. The information signal on the input screen-size issupplied to the code-amount assigning control circuit 24 of the codingsystem via an input terminal 56. The other constructions are the same asthose of the seventh preferred embodiment.

Similar to the ninth preferred embodiment, the tenth preferredembodiment of an image data coding system as shown in FIG. 19 has thescreen-size setting means 55 and the input terminal for inputting theinformation signal on the size of the received screen manually inputtedvia the input terminal 56. Since the other constructions are the same asthose of the eighth preferred embodiment as shown in FIG. 17, therepeated explanations are omitted.

As mentioned above, an image data coding system, according to thepresent invention, includes a screen-area determining means which canautomatically or manually set the area of the reproduced screen.Specifically, the screen-area determining means can analyze theresolution for analyzing the number of pixels, designate the size of thescreen by the header information, set the size of the screen in a manualoperation and so forth. Therefore, it is possible to improve the codingefficiency by reproducing the image after weighting in accordance withthe distribution of the closely observed points in view of human'svisual characteristic.

As mentioned above, an image data coding system, according to thepresent invention, was made by turning the inventor's attention that, inhuman's visual characteristic, the distribution of the closely observedpoints does not so diffuse when the object to be visually recognized issmall, and it is designed to vary the assignment of the code amount onthe respective regions on the screen without varying the code amount onthe whole reproduced screen when the size of the screen is small.Therefore, it is possible to subjectively improve the picture quality ofthe reproduced image.

Referring to the drawings, particularly to FIGS. 20 to 54, the preferredembodiments of an image data coding and/or decoding system, according tothe present invention, will be described below.

First Preferred Embodiment

FIG. 20 is a block diagram of the first preferred embodiment of an imagedata coding system, according to the present invention. An input imagesignal 10 is divided into a plurality of square blocks by a blockingcircuit (not shown) to be supplied to a substraction circuit 100. In thesubstraction circuit 100, a predicted error signal 30 which is adifference between a motion- compensated prediction signal supplied froma motion-compensated prediction circuit and the input image signal 10,is derived to be supplied to an orthogonal transform circuit 200.

The orthogonal transform circuit 200 transforms the predicted errorsignal 30 into an orthogonal transform coefficient in accordance with analpha map signal 20 supplied block by block, and then, supplies theorthogonal transform coefficient to a quantization circuit 120. Thequantized coefficient by the quantization circuit 120 is coded by avariable length coding circuit 140, as well as is inversely quantized byan inverse quantization circuit 130. An inversely quantized transformcoefficient 40 is inversely transformed by an inverse orthogonaltransform circuit 300, and then, added to the motion-compensatedprediction signal, which is supplied from a motion-compensatedprediction circuit 110, in an addition circuit 150.

A local-decoded image signal which is the output of the addition circuit150, is stored in a frame memory in the motion-compensated predictioncircuit 110. The transform coefficient coded by the variable lengthcoding circuit 140, and the alpha map signal coded by an alpha mapcoding circuit 160, together with side information such as motion-vectorinformation, are multiplexed in a multiplexed circuit 170 to beoutputted as a code bit stream 50. Furthermore, the alpha map signal iscoded by a method for coding a binary image, for example, by MMR(Modified Read).

The orthogonal transform circuit 200 and the inverse orthogonaltransform circuit 300 in FIG. 20 will be described in detail below.

FIGS. 21 and 22 are detailed block diagrams of the orthogonal transformcircuit 200 and the inverse orthogonal transform circuit 300,respectively.

The orthogonal transform circuit 200 as shown in FIG. 21 comprises aswitch circuit 210, an AS-DCT circuit 220 and a DCT circuit 230. Thealpha map signal 20 is supplied to both of the switch circuit 210 andthe AS-DCT circuit 220. The switch circuit 210 determines as to whetherthe block of the input predicted error signal 30 is an internal block,an external block or an edge block as shown in FIG. 53, to supply thepredicted error signal 30 to the DCT circuit 230 when it is the internalblock, and the predicted error signal 30 to the AS-DCT circuit 220 whenit is the edge block, i.e. the block containing the boundary portion ofa content. Furthermore, when it is the external block, the coding is notperformed or is performed by the other method.

FIG. 23 is a block diagram of the AS-DCT circuit 220, and FIG. 25 showsan example of a transform method in the AS-DCT. As shown in FIG. 25, thepixels contained in a content expressed by slanting lines in the inputedge block are first put together to the left end by a rearrangementcircuit 221. Then, in a DCT circuit 222, with respect to the pixelsexpressed by slanting lines, the one-dimensional DCT is performed in thehorizontal direction. Then, in a DCT circuit 224, the transformcoefficients expressed by a mesh are put together to the upper edge.Finally, in a DCT circuit 224, with respect to the transformcoefficients expressed by a mesh, the one-dimensional DCT is performedin the vertical direction. Furthermore, it is possible to change theorder of the rearrangement and DCT for processing.

The inverse orthogonal transform circuit 300 as shown in FIG. 22comprises a switch circuit 310, an AS-IDCT circuit 320 and an IDCTcircuit 330, and the alpha map signal 20 is supplied to both of theswitch circuit 310 and the AS-IDCT circuit 320.

FIG. 24 is a block diagram of the AS-IDCT circuit 320 which comprises anIDCT circuit 321, a rearrangement circuit 322, an IDCT circuit 323 and arearrangement circuit 324. Thus, in the inverse orthogonal transformcircuit 300, the operation contrary to the orthogonal transform circuit200 is performed.

An image data decoding system in this preferred embodiment will bedescribed below.

FIG. 26 is a block diagram of an image data decoding system having theresolution transform function corresponding to the image data codingsystem of FIG. 20. The input code bit stream 60 is separated into thecomponent of the transform coefficient and the alpha map signal in aseparating circuit 400. The code of the transform coefficient is decodedby a variable length decoding circuit 410, and then, is inverselyquantized by an inverse quantization circuit 420. On the other hand, thealpha map signal is decoded by an alpha map decoding circuit 430, andthen, is transformed into a desired resolution by a resolution transformcircuit 440.

The resolution transform circuit 440 performs the resolution transformof the alpha map signal which is a binary picture signal. As such amethod for performing the resolution transform of a binary picturesignal, for example, it is possible to use an enlargement and reductionmethod disclosed in "Image Processing Handbook" (p.630, Shokodo), whichwill be hereinafter referred to as "Literature 5". In a coefficientselecting circuit 450, the alpha map signal, the resolution of which istransformed in the resolution transform circuit 440, is rearranged inthe horizontal direction, and then, in the vertical direction, in thesame transform method as that of the aforementioned AS-DCT, as shown inFIG. 27. Furthermore, FIG. 27 is an example in which the resolution istransformed into 5/8 in both of the horizontal and vertical directions.

Then, the coefficient of a required band is selected from the transformcoefficients supplied by the inverse quantization circuit 420, to besupplied to an inverse orthogonal transform circuit 460. In the inversetransform circuit 460, with respect to the transform coefficient of theinternal block, the 5×5 of two-dimensional IDCT is performed, and withrespect to the transform coefficient of the edge block, the AS-IDCT isperformed in accordance with the resolution- transformed alpha mapsignal supplied by the resolution transform circuit 440, so that theinversely transformed signal is supplied to an addition circuit 470. Theaddition circuit 470 outputs a regenerative signal derived by adding amotion-compensated prediction signal supplied from a motion compensationcircuit 480 to a signal supplied from the inverse orthogonal transformcircuit 460.

Second Preferred Embodiment

Referring to FIGS. 28 to 34, the second preferred embodiment of an imagedata coding and/or decoding system, according to the present invention.

FIG. 28 is a block diagram of an image data coding system, according tothe present invention. In this embodiment, an orthogonal transformcircuit 250 and an inverse orthogonal transform circuit 350 comprise anAs-DCT circuit and an AS-IDCT circuit which can switch the orders of theAS-DCT circuit 220 of FIG. 21 and the AS-IDCT circuit 320 of FIG. 22,respectively. A correlation detecting circuit 180 detects a correlationbetween the components of the predicted error signal 30 in thehorizontal and vertical directions, and supplies a signal (a switchsignal) 21 indicative of the direction of the high correlation to theorthogonal transform circuit 250, the inverse orthogonal transformcircuit 350 and the multiplexer circuit 170. As a method for detectingthe correlation in the correlation detecting circuit 180, for example,there is a method for deriving the square error between the adjacentpixels in the horizontal and vertical directions.

FIGS. 29 and 30 are detailed block diagrams of the orthogonal transformcircuit 250 and the inverse orthogonal transform circuit 350,respectively. Similar to FIG. 21, the orthogonal transform circuit 250an shown in FIG. 29 comprises a switch circuit 210, an AS-DCT circuit260 and a DCT circuit 230. The alpha map signal 20 is supplied to theswitch circuit 210 and the AS-DCT circuit 260, and the switching signal21 is supplied to the AS-DCT circuit 260. The inverse orthogonaltransform circuit 350 as shown in FIG. 30 comprises a switch circuit310, an AS-IDCT circuit 360 and an IDCT circuit 330. The alpha mapsignal 20 is supplied to the switch circuit 310 and the AS-IDCT circuit360, and the switching signal 21 is supplied to the IDCT circuit 330.

FIGS. 31 and 32 are block diagrams of the AS-DCT circuit 260 of FIG. 29and the AS-IDCT circuit 360 of FIG. 30, respectively. FIG. 33 is a viewexplaining, in detail, a method for switching the order of transform inthe AS-DCT circuit 260 and the AS-IDCT circuit 360. The order oftransform is changed by switching first switch circuits 261, 361 andsecond switch circuits 262, 362 of FIGS. 31 and 32, in the manner asshown in FIGS. 33(a) and 33(b). Specifically, by means of the switchingsignal 21, the switch circuits 261, 262 are switched as shown in FIG.33(a) when the correlation in the horizontal direction is high, and asshown in FIG. 33(b) when the correction in the vertical direction ishigh. Furthermore, the switching signal 21 may be coded with one bitblock by block or with one bit by frame by frame.

An image data decoding system, according to the present invention, willbe described below.

FIG. 34 is a block diagram of an image data decoding system having theresolution transform function corresponding to the image coding systemof FIG. 28. The point different from the first preferred embodiment ofan image data decoding system as shown in FIG. 26 is that a transformorder switching signal 61 separated from an input coding bit stream 60in the separating circuit 400 is supplied to an inverse orthogonaltransform circuit 461. The inverse orthogonal transform circuit 461 isthe same as the inverse orthogonal transform circuit 350 in the imagedata coding system as shown in FIG. 30, and switches the order oftransform by the switching signal 61 in the same manner as thatdescribed in FIG. 33.

Referring to FIGS. 35 and 36, an example of a method for scanning thetransform coefficient of the AS-DCT will be described below. In general,in a case where the DCT coefficient of a square block is coded, afterzigzag scan, the synthesized phenomenon of the magnitude of thecoefficient and the zero run length is coded using the two-dimensionalvariable length coding (see "Image Coding Techniques--DCT and ItsInternational Standard--", pp288-290). On the other hand, in the AS-DCT,due to the shape of the block, the distribution of the transformcoefficient is leaned in the horizontal direction h and the verticaldirection v as shown in FIG. 35. Therefore, according to this preferredembodiment, in both of the image data coding system and the image datadecoding system, the order of scan is determined so as to adapt to thedistribution of transform coefficients which can be specified by thealpha map signal.

FIG. 35 is an example of a method for determining the order of scan.First, the alpha map signal 20 (0: Outside of Content, 1: Inside ofContent) is rearranged in the horizontal direction h and the verticaldirection v to derive the distribution of transform coefficients (map[v] [h]: v, h=0˜size-1). Then, in accordance with a method described byC-Language as follows, the order of scan (order [v] [h]:v, h=˜size-1) isdetermined. ##EQU1##

In addition, various scan methods for actualizing scalable function oncoding data has been proposed (see "Image Coding Techniques", FIG.7.114).

FIG. 36 is a block diagram of a scan-method determining circuit foractualizing scalable function in the AS-DCT. In the resolution transformcircuit 440, the resolution transform of an alpha map signal 80 isperformed (for example, 1/2 in both of the horizontal and verticaldirections) to be supplied to a first horizontal and verticalrearrangement circuit 441. To a second horizontal and verticalrearrangement circuit 442, the alpha map signal 80, the resolutiontransform of which is not performed, is supplied. In the horizontal andvertical rearrangement circuits 441, 442, the rearrangement as shown inFIG. 35 is performed. As a result, (map [v] [h]) is supplied to a firstscan-order determining circuit 444 and an exclusive OR operation circuit443.

In the first scan-order determining circuit 444, the scan order (theorder of a low-band component) of the resolution transformed alpha mapsignal is determined in a manner of FIG. 35, and the information 81indicative of the scan order is supplied to a second scan-orderdetermining circuit 445. The exclusive OR operation circuit 443 derivesa difference between the map [v] [h] of a low resolution supplied fromthe first horizontal and vertical rearrangement circuit 441 and the map[v] [h] of a high resolution supplied from the second horizontal andvertical rearrangement circuit 442, and supplies this difference 82 tothe second scan-order determining circuit 445.

In the second scan-order determining circuit 445, the scan order of ahigh-band component is determined subsequently to the scan order of thelow-band component determined by the first scan-order determiningcircuit 444, and the information 83 indicative of the scan order of thecombination of the low and high band components is output. Thisalgorithm is applicable when more multistage division is performed todetermine the scan order.

Third Preferred Embodiment

Referring to FIGS. 37 to 40, the third preferred embodiment of an imagedata coding and/or decoding system, according to the present invention,will be described below.

FIG. 37 is a block diagram of an image data coding system in thisembodiment. In an average value separating circuit 500, if the predictederror signal 30 is an edge block signal in accordance with the alpha mapsignal 20, an average value of the signals inside of a content (theportion of oblique lines in FIG. 38) is derived to be separated, and allthe signals outside of the content is set to be zero. By thisprocessing, the average value in the central square blocks of FIG. 38becomes zero. When the signal 32 indicative of the average value 0 inthe blocks is supplied to the DCT circuit 230 for performing thetwo-dimensional DCT, the DC component becomes zero as the right-sidesquare blocks of FIG. 38. In this case, an extrapolated signal may besubstituted for the signals outside of the content under the conditionthat the average value is 0.

The average value 31 derived in the average value separating circuit500, together with the alternating current transform coefficient of theDCT supplied from the DCT circuit, is supplied to a quantization circuit121, and quantized to be supplied to an inverse quantization circuit 131and a variable length coding circuit 140. In the inverse quantizationcircuit 131, the average value and the alternating current transformcoefficient are inversely quantized. A quantized average value 41 issupplied to an average value synthesizing circuit 510, and a quantizedalternating current transform coefficient 42 is supplied to the inverseDCT circuit 330.

In the average value synthesizing circuit 510, a regenerative signal isderived by synthesizing the signals inside of the content with theaverage value 41 in accordance with the alpha map signal 20 inverselytransformed in the inverse DCT circuit 330. At this time, the signalsoutside of the content are reset to be, for example, zero.

An image data decoding system in this embodiment will be describedbelow.

FIG. 39 is a block diagram of an image data decoding system having theresolution transform function corresponding to the image data codingsystem of FIG. 37, and FIG. 40 is a view showing a method forreproducing the signal that the resolution transform is performed. InFIG. 39, an inverse quantization circuit 421 inversely quantizes theaverage value and the alternating current transform coefficient, tosupply an average value 62 to an average value synthesizing circuit 511and an alternating current transform coefficient 63 to a coefficientselecting circuit 451, respectively. In an inverse DCT circuit 462, theDCT is performed with respect to the transform coefficient having a bandnecessary to derive a desired resolution selected in the coefficientselecting circuit 451 (in an example of FIG. 40, a 5×5 oftwo-dimensional IDCT).

In the average value synthesizing circuit 511, a regenerative signal isderived by synthesizing the signal inversely transformed by the inverseDCT circuit 462 with the average value 62 in the signals inside of thecontent, in accordance with the resolution-transformed alpha map signalsupplied from the resolution transform circuit 440.

Fourth Preferred Embodiment

Referring to FIGS. 41 to 44, the fourth preferred embodiment of an imagedata coding and/or decoding system, according to the present inventionwill be described below.

FIG. 41 is a block diagram of an image data coding system in thisembodiment. In an average value deriving circuit 501, when the predictederror signal 30 is the edge block signal in accordance with the alphamap signal 20, the average value a of the pixels inside of the content(the portion expressed by the oblique lines in FIG. 42) is derived to besupplied to an average value inserting circuit 502. In this averagevalue inserting circuit 502, as shown in FIG. 42, the processing forassuming all the values of the pixels outside of the content to be theaverage value a of the pixels inside of the content is performed (theinsertion of the average value). By this processing, the intrablockaverage value in the square block at the center of FIG. 42 becomes a.When the signal of this intrablock average value a is supplied to theDCT circuit 230 for performing the two-dimensional DCT, the DC componentbecomes A (=8×a) as the square block on the right-side of FIG. 42. Atthis time, an extrapolated signal may be substituted for the signaloutside of the content under the condition that the average value is a.

The output of the average value inserting circuit 502 is supplied to theDCT circuit 230 to be transformed into a DCT coefficient, and then, issupplied to the quantization circuit 120 to be quantized therein. Thequantized transform coefficient is supplied to the inverse quantizationcircuit 130 and the variable length coding circuit 140. In the inversequantization circuit 130, the transform coefficient supplied by thequantization circuit 120 is inversely transformed to be supplied to theinverse DCT circuit.

In a pixel separating circuit 512, a regenerative picture signal isderived by separating the signals indicative of the pixels inside of thecontent from the signals inversely transformed in the inverse DCTcircuit 330, in accordance with the alpha map signal 20. At this time,the signals outside of the content are reset to be zero for example.

An image data decoding system in this embodiment will be describedbelow. FIG. 43 is a block diagram of an image data decoding systemhaving the resolution transform function corresponding to the image datacoding system of FIG. 41, and FIG. 44 is a view showing a method forreproducing a resolution-transformed signal. In FIG. 43, the transformcoefficient is inversely transformed by the inverse quantization circuit420 to be supplied to the coefficient selecting circuit 451. In theinverse DCT circuit 462, the DCT is performed with respect to thetransform coefficient of a band required to derive a desired resolutionselected by the coefficient selecting circuit 451 (in the example ofFIG. 44, a 5×5 of two-dimensional IDCT).

In a pixel separating circuit 513, a regenerative signal is derived byseparating the signal inversely transformed by the inverse DCT circuitfrom the signals indicative of the pixels inside of the content, inaccordance with the resolution-transformed alpha map signal suppliedfrom the resolution transform circuit 440.

Fifth Preferred Embodiment

Referring to FIG. 45 to 49, the fifth preferred embodiment of an imagedata coding and/or decoding system, according to the present invention,will be described below.

In this embodiment, a method for coding a block of an optional shape bythe vector quantization (VQ) is used. FIG. 41 is a block diagram of animage data coding system in this embodiment, and FIG. 42 is a viewshowing a method for coding in an edge block.

In FIG. 45, a vector quantizer 600 performs the matching of thepredicted error signal 30 with code vectors stored in a code book, toselect a code vector of the highest correlation to the predicted errorsignal 30. At this time, as shown in FIG. 46, with respect to the edgeblock, the matching of only the signals inside of the content (theportion expressed by the oblique lines in the drawing) with the codevectors is performed in accordance with the alpha map signal 20, tooutput an index of the code vector of the highest correlation ("2" in anexample of FIG. 46).

In the coding circuit 141, the index supplied by the vector quantizer600 is coded with a variable length or a fixed length to be outputted tothe multiplexer circuit 170. In an inverse vector quantizer 610, asshown in FIG. 47, the signals inside of the content (the portionexpressed by the oblique lines in the drawings) are separated from thecode vector corresponding to the index supplied from the vectorquantizer 600 for outputting a regenerative signal.

An image data decoding system in this embodiment will be describedbelow.

FIG. 48 is a block diagram of an image data decoding system having theresolution transform function corresponding to the image data codingsystem of FIG. 45, and FIG. 49 shows code blocks provided in an inversevector quantizer 620. The alpha map signal separated by the separatingcircuit 400 from the input coding bit stream 60 is decoded by the alphamap decoding circuit 430, to be resolution-transformed into theresolution of each of picture signals by the resolution transformcircuit 440. On the other hand, the index separated by the separatingcircuit 400 is decoded by the decoding circuit 411 to be supplied to theinverse vector quantizer 620.

In the inverse vector quantizer 620, as shown in FIG. 29, a code vectorof a desired resolution is selected from code vectors expressed bymultiple resolution corresponding to the index, and the signal inside ofthe content (the portion expressed by the oblique lines in the drawing)is separated in accordance with the resolution-transformed alpha mapsignal supplied from the resolution transform circuit 440. On the basisof this signal, a regenerative signal 70 is derived by the additioncircuit 470 and the motion compensating circuit 480.

Sixth Preferred Embodiment

Referring to FIGS. 50 to 53, the sixth preferred embodiment of an imagedata coding and/or decoding system, according to the present invention,will be described below.

FIGS. 50 and 52 are views explaining the subband division of a picturesignal. The subband division of the input picture signal is performed bythe band division and the down sampling. FIG. 50 shows an example thatthe subband division is performed by dividing the input image into fourbands (LL, LH, HL, HH) or further dividing the band LL into four bandsto derive seven bands. FIG. 51 shows the arrangement of each ofcomponents on the axes of space frequencies when the subband divisioninto four bands is performed. An example of the subband division of theinput picture signal into four bands will be described below.

FIG. 52 is a block diagram of an image data coding system in thisembodiment. The input picture signal 10 is divided into a plurality ofsubband picture signals in a subband division circuit 700, to beinputted to optional-shape coding circuits 710, 711, 712 and 713. Thesubband picture signals LL, LH, HL and HH are coded in theoptional-shape coding circuits 710, 711, 712 and 713, respectively, inan optional-shape coding method which is the same manner as thatdescribed in any one of the first to fourth preferred embodiment. Atthis time, the alpha map signals are transformed to the resolution ofeach of the subband images by a resolution transform circuit 446, to besupplied to the optional shape coding circuits 710, 711, 712 and 713,respectively. The coded alpha map signals and the subband signals areoutputted as a coding bit stream 50 through the multiplexer circuit 170.

FIG. 53 is a block diagram of an image data decoding system, in thisembodiment, adapted to the image data coding system of FIG. 52. Thealpha map signal separated by the separating circuit 400 from the inputcoding bit stream 60 is decoded by the alpha map decoding circuit 430,and the resolution transform thereof into the resolution of each of thesubband picture signals is performed by the resolution transform circuit446.

On the other hand, the subband picture signals separated by theseparating circuit 400 are inputted to the optional-shape decodingcircuits 720, 721, 722 and 723. In accordance with the alpha map signalssupplied from the resolution transform circuit 446, the subband picturesignals LL, LH, HL and HH are reproduced in the optional-shape decodingcircuits 720, 721, 722 and 723, respectively, in the same optional-shapedecoding method as that described in each of the first to fourthpreferred embodiments. That is, for example, with respect to the edgeblock, only the subband image signals inside of the content are decoded.

Each of the reproduced subband picture signals are outputted as aregenerative picture signal 70 after synthesizing only the subbandsignals necessary to derive a predetermined resolution in a subbandsynthesizing circuit 730. For example, if only the subband image LL isoutputted as the reproduced picture signal 70, the image of a lowresolution is reproduced.

Referring to FIG. 54, as an applied embodiment of the present invention,the preferred embodiment of an image transmitting system to which animage data coding and/or decoding system of the present invention isapplied, will be described below.

The picture signal inputted by a camera 1002 mounted on a personalcomputer (PC) 1001 is coded by an image data coding system installed inthe PC 1001. The coding data outputted from this image data codingsystem is multiplexed with the information on other voice and data, tobe sent by a wireless installation 1003 and received by another wirelessinstallation 1004. The signal received by the wireless installation 1004is analyzed into the coding data of the picture signals, and theinformation on voice and data. Among them, the coding data of thepicture signals are decoded by an image data decoding system installedin a work station (EWS) 1005, to be indicated on a display of the EWS1005.

On the other hand, the picture signals inputted by a camera 1006 mountedon the EWS 1005 is coded using an image data coding system installed inthe EWS, in the same manner as that of the aforementioned manner. Thecoding data are multiplexed with the other information on voice anddata, to be sent by the wireless installation 1004 and received by thewireless installation 1003. The signals received by the wirelessinstallation 1003 are analyzed into the coding data of the picturesignals, and the information on voice and data. Among them, the codingdata of the picture signals are decoded by an image data decoding systeminstalled in the PC 1001 to be indicated on a display of the PC 1001.

Furthermore, the sending and receiving of data can be performed using awire transmitting system, not wireless transmitting system.

As mentioned above, according to the present invention, it is possibleto perform the resolution transform of an edge block containing acontent of an optional shape, and it is also possible to code the edgeblock without reducing the coding efficiency compared with conventionalcoding methods.

Other embodiments of the invention are apparent to those skilled in theart from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only with a true scope and spirit ofthe invention being indicated by the following claims.

What is claimed is:
 1. An image data coding system comprising:firstcoding means for coding a map signal indicative of a position and shapeof a content on a display area of picture signals to be coded;resolution transform means for transforming a resolution of said mapsignal; orthogonal transform means for performing a first orthogonaltransform and a two dimensional orthogonal transform, different fromsaid first orthogonal transform, of said picture signals in accordancewith said resolution after transformed of the map signal to output anorthogonal transform coefficient; and second coding means for codingsaid orthogonal transform coefficient derived by said orthogonaltransform means, wherein said orthogonal transform means performs thetwo-dimensional orthogonal transform of the picture signals of allpixels for blocks located inside a boundary portion of the content, andperforms the orthogonal transform of at least the picture signals of thepixels for blocks containing the boundary portion of the content.
 2. Animage data decoding system comprising:first decoding means for decodinga coded map signal indicative of a position and shape of a content on adisplay area inputted for every block of picture signals; resolutiontransform means for performing a resolution transform of the map signaldecoded by said first decoding means; second decoding means for decodingcoded orthogonal transform coefficients; coefficient selecting means forselecting an orthogonal transform coefficient necessary to reproduce animage of a predetermined resolution, from the orthogonal transformcoefficients decoded by said second decoding means, based on the mapsignal resolution-transformed by said resolution transform means;inverse orthogonal transform means for performing an inverse orthogonaltransform of the orthogonal transform coefficient selected by saidcoefficient selecting means; and reproducing means for deriving aregenerative picture signal resolution-transformed from the results ofthe inverse orthogonal transform by said inverse orthogonal transformmeans, wherein said inverse orthogonal transform means performs atwo-dimensional inverse orthogonal transform of all the coefficients forblocks located inside a boundary portion of the content among theorthogonal transform coefficients selected by said coefficient selectingmeans, and performs the inverse orthogonal transform of only thecoefficients for blocks containing the boundary portion of the content.3. An image data coding system comprising:first coding means for codinga map signal indicative of a position and shape of a content on adisplay area of picture signals to be coded; resolution transform meansfor transforming a resolution of said map signal; average valueseparating means for outputting values of pixels by outputting anaverage value of pixels when blocks include a boundary portion of thecontent, outputting a difference value between the average value and apixel value when blocks are located inside the boundary portion of thecontent, and outputting a zero value when blocks are located outside theboundary portion of the content in accordance with said transformedresolution of said map signal; orthogonal transform means for performinga two-dimensional orthogonal transform of the signals from which theaverage value has been separated by said average value separating means,in order to output orthogonal transform coefficients; and second codingmeans for coding said orthogonal transform coefficients outputted bysaid orthogonal transform means and said average value.
 4. An image datadecoding system comprising:first decoding means for decoding a coded mapsignal indicative of the position and shape of a content on a displayarea inputted for every block of picture signals; resolution transformmeans for performing a resolution transform of the map signal decoded bysaid first decoding means; second decoding means for decoding codedorthogonal transform coefficients and an average value of pixels insideof said content; coefficient selecting means for selecting an orthogonaltransform coefficient necessary to reproduce an image of a predeterminedresolution, from the orthogonal transform coefficients decoded by saidsecond decoding means, based on the map signal resolution-transformed bysaid resolution transform means; inverse orthogonal transform means forperforming a two-dimensional inverse orthogonal transform of theorthogonal transform coefficient selected by said coefficient selectingmeans; and reproducing means for deriving a resolution-transformedregenerative picture signal by synthesizing the results of said inverseorthogonal transform means with said average value decoded by saidsecond decoding means, based on said map signal resolution-transformedby said resolution transform means.
 5. An image data coding systemcomprising:first coding means for coding a map signal indicative of aposition and shape of a content on a display area of picture signals tobe coded; resolution transform means for transforming a resolution ofsaid map signal; average value inserting means for replacing the valuesof pixels located outside a boundary portion of the content with anaverage value signal of the values of pixels located inside the boundaryportion of content, in accordance with said resolution after transformedof the map signal; orthogonal transform means for performing atwo-dimensional orthogonal transform of the average value signal in theblocks produced by said average value inserting means, in order tooutput orthogonal transform coefficients; and second coding means forcoding said orthogonal transform coefficients outputted by saidorthogonal transform means.
 6. An image data decoding systemcomprising:first decoding means for decoding a coded map signalindicative of a position and shape of a content on a display areainputted for every square block of picture signals; resolution transformmeans for performing a resolution transform of the map signal decoded bysaid first decoding means; second decoding means for decoding codedorthogonal transform coefficients; coefficient selecting means forselecting an orthogonal transform coefficient necessary to reproduce animage of a predetermined resolution, from the orthogonal transformcoefficients decoded by said second decoding means, based on the mapsignal resolution-transformed by said resolution transform means;inverse orthogonal transform means for performing a two-dimensionalinverse orthogonal transform of the orthogonal transform coefficientselected by said coefficient selecting means; and reproducing means forderiving a resolution-transformed regenerative picture signal by takingout the values of pixels located inside a boundary portion of thecontent from the results of the two-dimensional inverse orthogonaltransform by said inverse orthogonal transform means, based on the mapsignal resolution-transformed by said resolution transform means.
 7. Animage data coding system comprising:first coding means for coding a mapsignal indicative of a position and shape of a content on a display areaof picture signals to be coded; resolution transform means fortransforming a resolution of said map signal; vector quantizing meansfor quantizing said picture signals by matching said picture signalswith code vectors stored in a code book so as to output an indexindicative of a code vector having the highest correlation to saidpicture signals when blocks are located inside a boundary portion of thecontent, and matching at least the picture signals inside the boundaryportion of the content with said code vectors when blocks contain theboundary portion of the content in accordance with said resolution aftertransformed of the map signal; and second coding means for coding saidindex outputted by said vector quantizing means.
 8. An image datadecoding system comprising:first decoding means for decoding a coded mapsignal indicative of a position and shape of a content on a display areainputted for every block of picture signals to be coded; resolutiontransform means for performing a resolution transform of the map signaldecoded by said first decoding means; second decoding means for decodinga coded index; and inverse vector quantizing means, having a code bookthat stores code vectors indicated by multiple resolutions, foroutputting a code vector of a desired resolution designated by the indexdecoded by said second decoding means when blocks are located inside aboundary portion the content, and deriving a resolution-transformedregenerative picture signal by taking out only the signal located insidethe boundary portion of the content, when the blocks contain theboundary portion of the content, from the code vectors according to themap signal resolution-transformed by said resolution transform means. 9.An image data coding system comprising:first coding means for coding amap signal indicative of a position and shape of a content on a displayarea of picture signals to be coded; subband dividing means for dividingsaid picture signal into a plurality of subband picture signals;resolution transform means for performing a resolution transform of saidmap signal into the resolution of each of the subband picture signalsdivided by said subband dividing means; and second coding means forcoding each of the subband picture signals divided by said subbanddividing means when blocks are located inside a boundary portion of thecontent, and coding at least the signals located inside the boundaryportions of the content when the blocks contain the boundary portion ofthe content in said subband picture signals according to the map signalresolution-transformed by said resolution transform means.
 10. An imagedata decoding system comprising:first decoding means for decoding acoded map signal indicative of a position and shape of a content on adisplay area of picture signals to be coded; resolution transform meansfor performing a resolution transform of the map signal decoded by saidfirst decoding means, into the resolutions of a plurality of subbandpicture signals; second decoding means for decoding a plurality of codedsubband signals in accordance with the map signal resolution transformedby said resolution transform means; and subband synthesizing means forderiving a resolution-transformed regenerative picture signal bysynthesizing at least the subband picture signals necessary to reproducean image of a predetermined resolution among the plurality of subbandpicture signals decoded by said second decoding means.